{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6d89f34a508ac4a0",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Bag-of-Words词袋\n",
    "\n",
    "词袋技术是一种将文本转换为向量的一种方法。\n",
    "\n",
    "作用：用来将文字表示为向量\n",
    "\n",
    "特点：忽略词和词之间的顺序，从而忽略了语法、上下文\n",
    "\n",
    "流程：\n",
    "1. 先分词，并把词放入“袋子”中，“袋子”中的词是唯一的\n",
    "2. 再创建词表，给“袋子”中的每个词设置一个编号\n",
    "3. 依次把文章中的每个句子转成向量，每个句子对应一个向量，向量长度为词表长度，向量的每个位置对应了词表中的每个词，一个句子中有哪些词，就在对应向量中的位置设置为词出现的次数（难理解，看代码容易理解）\n",
    "4. 最终得到的向量就是句子对应的向量，叫做Bag of Words向量"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfa8a4b7b957279b",
   "metadata": {},
   "source": [
    "### 准备语料"
   ]
  },
  {
   "cell_type": "code",
   "id": "4e590b47dc027049",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:36:46.716637Z",
     "start_time": "2025-07-02T01:36:46.710387Z"
    }
   },
   "source": [
    "# 语料\n",
    "texts = [\n",
    "    \"I love natural language processing.\",\n",
    "    \"I love machine learning.\",\n",
    "    \"I love coding in Python and Java.\",\n",
    "    \"I love Java.\",\n",
    "    \"I love Java, I don't love C++\",\n",
    "    \"I don't love Java.\"\n",
    "]"
   ],
   "outputs": [],
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "id": "5540bd12843c638c",
   "metadata": {},
   "source": [
    "### 分词"
   ]
  },
  {
   "cell_type": "code",
   "id": "be722d65afa67e12",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:36:47.966722Z",
     "start_time": "2025-07-02T01:36:47.964158Z"
    }
   },
   "source": [
    "# 分词\n",
    "words = [word for text in texts for word in text.split()]\n",
    "print(words)"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['I', 'love', 'natural', 'language', 'processing.', 'I', 'love', 'machine', 'learning.', 'I', 'love', 'coding', 'in', 'Python', 'and', 'Java.', 'I', 'love', 'Java.', 'I', 'love', 'Java,', 'I', \"don't\", 'love', 'C++', 'I', \"don't\", 'love', 'Java.']\n"
     ]
    }
   ],
   "execution_count": 4
  },
  {
   "cell_type": "markdown",
   "id": "dd6ef0379eb2ebe5",
   "metadata": {},
   "source": [
    "### 构造词汇表"
   ]
  },
  {
   "cell_type": "code",
   "id": "90eb9af5bc298ae0",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:41:00.183443Z",
     "start_time": "2025-07-02T01:41:00.179847Z"
    }
   },
   "source": [
    "# 构造词汇表\n",
    "vocabulary = {}\n",
    "for word in words:\n",
    "    if word not in vocabulary:\n",
    "        vocabulary[word] = len(vocabulary)\n",
    "\n",
    "vocabulary"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'I': 0,\n",
       " 'love': 1,\n",
       " 'natural': 2,\n",
       " 'language': 3,\n",
       " 'processing.': 4,\n",
       " 'machine': 5,\n",
       " 'learning.': 6,\n",
       " 'coding': 7,\n",
       " 'in': 8,\n",
       " 'Python': 9,\n",
       " 'and': 10,\n",
       " 'Java.': 11,\n",
       " 'Java,': 12,\n",
       " \"don't\": 13,\n",
       " 'C++': 14}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 6
  },
  {
   "cell_type": "markdown",
   "id": "b0993f0bc7f1669b",
   "metadata": {},
   "source": [
    "### 转成bag-of-words向量"
   ]
  },
  {
   "cell_type": "code",
   "id": "338979aa23d2f31",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:41:01.909451Z",
     "start_time": "2025-07-02T01:41:01.905160Z"
    }
   },
   "source": [
    "# 将原始句子根据词汇表构造为一个向量，称为bag-of-words向量\n",
    "# 每个向量长度一样，都等于词汇表的长度\n",
    "# \"I love natural language processing.\",\n",
    "bows = []\n",
    "for text in texts:\n",
    "    bow = [0] * len(vocabulary)   # 词表长度的全零向量\n",
    "    for word in text.split():\n",
    "        bow[vocabulary[word]] += 1\n",
    "    bows.append(bow)\n",
    "\n",
    "bows"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       " [1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],\n",
       " [1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0],\n",
       " [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0],\n",
       " [2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1],\n",
       " [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0]]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 7
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "\"I love Java, I don't love C++\",",
   "id": "fbc98ff830448c6"
  },
  {
   "cell_type": "markdown",
   "id": "b7d8f2929bc1105d",
   "metadata": {},
   "source": [
    "## 相似度\n",
    "\n",
    "两个句子的向量相似，就代表两个句子中的词相似，就代表两个句子的语义相似\n",
    "\n",
    "### 余弦相似度（Cosine Similarity）\n",
    "- **原理**：计算两个向量的夹角余弦值（忽略向量长度）。\n",
    "- **公式**：\n",
    "$$cos(\\theta) = \\frac{\\mathbf{A} \\cdot \\mathbf{B}}{\\|\\mathbf{A}\\| \\cdot \\|\\mathbf{B}\\|} = \\frac{\\sum_{i=1}^{n} A_i B_i}{\\sqrt{\\sum_{i=1}^{n} A_i^2} \\cdot \\sqrt{\\sum_{i=1}^{n} B_i^2}}$$\n",
    "- **应用**：文本相似度、推荐系统\n",
    "\n",
    "### 点积相似度（Dot Product Similarity）\n",
    "- **原理**：计算向量的内积（方向 + 长度均考虑）。\n",
    "- **公式**：\n",
    "$${sim}(\\mathbf{A}, \\mathbf{B}) = \\mathbf{A} \\cdot \\mathbf{B} = \\sum_{i=1}^{n} A_i B_i$$\n",
    "- **应用**：Transformer"
   ]
  },
  {
   "cell_type": "code",
   "id": "e63a91b9de6c7f08",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:48:16.478882Z",
     "start_time": "2025-07-02T01:48:16.475187Z"
    }
   },
   "source": [
    "# 计算两个向量的相似度\n",
    "def cosine_similarity(v1, v2):\n",
    "    dot_product = sum(v1[i] * v2[i] for i in range(len(v1)))\n",
    "    norm_v1 = sum(v1[i] ** 2 for i in range(len(v1))) ** 0.5\n",
    "    norm_v2 = sum(v2[i] ** 2 for i in range(len(v2))) ** 0.5\n",
    "    return dot_product / (norm_v1 * norm_v2)\n",
    "\n",
    "cosine_similarity([1, 2, 3], [1, 2, 6])"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9600014517991345"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 19
  },
  {
   "cell_type": "markdown",
   "id": "4e2dffd7fb80a7f4",
   "metadata": {},
   "source": [
    "### 为什么说忽略了向量长度\n",
    "公式中的$\\|\\mathbf{A}\\|$和$\\|\\mathbf{B}\\|$分别表示向量A和B的模长，向量A除以它的模长，向量B除以它的模长，相当于做了归一化，相当于忽略了向量长度。"
   ]
  },
  {
   "cell_type": "code",
   "id": "b44b4ffcc36c9a6f",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T02:01:17.812533Z",
     "start_time": "2025-07-02T02:01:17.809484Z"
    }
   },
   "source": [
    "# 方向一致，长度相差很大\n",
    "cosine_similarity([1, 1, 0], [100, 100, 0])"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9999999999999999"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 45
  },
  {
   "cell_type": "code",
   "id": "959c85a5313ccee0",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T02:01:27.125293Z",
     "start_time": "2025-07-02T02:01:27.121655Z"
    }
   },
   "source": [
    "# 长度一致，方向相差大\n",
    "cosine_similarity([1, 1, 0], [0, 1, 1])"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4999999999999999"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 46
  },
  {
   "cell_type": "markdown",
   "id": "691c40c741935626",
   "metadata": {},
   "source": [
    "### 测试语义相似度"
   ]
  },
  {
   "cell_type": "code",
   "id": "959b74e49af0250",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:49:11.607743Z",
     "start_time": "2025-07-02T01:49:11.603307Z"
    }
   },
   "source": [
    "print(texts[2])\n",
    "print(texts[3])\n",
    "cosine_similarity(bows[2], bows[3])"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I love coding in Python and Java.\n",
      "I love Java.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.6546536707079772"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 22
  },
  {
   "cell_type": "markdown",
   "id": "9ca787cb94f94f3e",
   "metadata": {},
   "source": [
    "### Bag-of-Words缺点"
   ]
  },
  {
   "cell_type": "code",
   "id": "4c8d7745db70bdc7",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-07-02T01:50:09.024001Z",
     "start_time": "2025-07-02T01:50:09.021114Z"
    }
   },
   "source": [
    "print(texts[3])\n",
    "print(texts[5])\n",
    "cosine_similarity(bows[3], bows[5])"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I love Java.\n",
      "I don't love Java.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.8660254037844387"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 23
  },
  {
   "cell_type": "markdown",
   "id": "a06f4af2fe4ae3b4",
   "metadata": {},
   "source": [
    "可以看出，\"I love Java.\"和\"I don't love Java.\"语义是完全相反的，但是相似度却很高，这就是词袋模型的缺点。\n",
    "\n",
    "另外，词汇表越大，那么bow向量也就越长，这也是词袋模型的缺点。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
